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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )S Responsive to communication(s) filed on 01 September 2005 . 
2a)Q This action is FINAL. 2b)K This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) [3 Claim(s) 1-18 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) 03 Claim(s) 1-18 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) Q Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .1 21 (d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 
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1 0 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
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DETAILED ACTION 
THIS IS NON-FINAL OFFICE ACTION TO THE PENDING CLAIMS 1-18 

1 . In view of the appeal Brief filed on 971 /2005, PROSECUTION 

IS HEREBY REOPENED. A new rejection to the pending claims 1-18 is set forth below. 

To avoid abandonment of the application, appellant must exercise one of the 
following two options: 

(1 ) file a reply under 37 CFR 1 .1 1 1 (if this Office action is non-final) or a reply 
under 37 CFR 1 .1 13 (if this Office action is final); or, 

(2) request reinstatement of the appeal. 

If reinstatement of the appeal is requested, such request must be accompanied 
by a supplemental appeal brief, but no new amendments, affidavits (37 CFR 1 .130, 
1.131 or 1.132) or other evidence are permitted. See 37 CFR 1.193(b)(2). 

2. Examiner acknowledges applicant's Appeal Brief filed on 9/1/2005. 

Drawings 

3. The drawings filed on 2/7/2002 is acceptable for examination purpose. 

Priority 

4. Applicant's claim for domestic priority under 35 U.S.C, 1 19(e) is acknowledged 
based on the provisional application SI.No. 60/329,770 filed on 10/16/2001 

5. Examiner notes that applicant filed international application no. PCT/US02/32980 
on 10/16/2002 is now published WO 03/034284A1. 



Application/Control Number: 10/071 ,731 Page 3 

Art Unit: 2166 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 1-8,10-18 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Breese et al. [hereafter Breese], US Patent No. 5,987,415 filed on Ju 30,1998, 
published on Nov 16, 1999 in view of Mizokawa, US Patent No.6, 230,111 filed on 
Aug 6,1998, published on May 8, 2001 

7. As to Claims 1,8 12,18 Breese teaches a system which including •method of 
obtaining information regarding an environment for an individual having preferred 
modalities and engaged in activity using a programmable device 1 [see Abstract, col 4, 
line 57-62, col 6, line 36-44,col 8, line 8-12, col 10, line 23-27, fig 3], environment for an 
individual corresponds to Breese 's computer user interface that including observing 
user behavior, particularly user behavior caused from emotion and personality state as 
detailed in col 4, line 59-60, further it is noted that Breese specifically teaches a model 
that influence of emotion and personality based on Bayesian network as detailed in 

fig 3, programmable device corresponds to program modules being executed by a 
personal computer because program modules include processes, programs and like as 
detailed in col 6, line 40-42; 'sensing at least one psychomotor behavioral element of 
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the activity engaged by the individual' [col 8, line 23-28, line 35-41, col 10, line 34-42], 
Breese specifically teaches Bayesian net integrate various aspects of emotion and 
personality in a single model for example cognitive response as well as physical 
response based on the metal state, further, it is noted that personality nodes capture or 
senses individual emotional and or personality state as detailed in col 10, line 34-42; 
'psychomotor behavior element of the activity engaged by the individual' 
[col 12, line 49-59, fig 6], psychomotor behavior elements corresponds to loud, angry 
voice tones, calm, quite voice related to emotional state and personality of the user as 
detailed in col 12, line 50-52. It is however, noted that Breese does not specifically 
teach "determining the preferred modalities of the individual'. On the other hand, 
Mizokawa teaches a system which including 'determining the preferred modalities of the 
individual' [col 6, line 55-60], preferred modalities of the individual corresponds to 
Mizokawa's user's emotions as detailed in col 6, line 58-60. 

It would have been obvious to one of the ordinary skill in the art at the time of 
applicant's invention to incorporate the teachings of Mizokawa into modeling user's 
emotion and personality in a computer user interface of Breese et al. because both 
Breese and Mizokawa are directed to user behavior and emotion model, more 
specifically Breese is directed to computer based capturing emotion and personality 
states that including multistage representation of emotional and personality variables 
[see Abstract, col 5, line 22-28], while Mizokawa is directed to controlling object using 
pseudo-emotions and pseudo-personality generated in the object, more specifically 
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recognizing, updating user commands related to pseudo-personality, pseudo-emotions 
and pattern of behavior [see Abstract, fig 2-3]. 

One of the ordinary skill in the art at the time of applicant's invention to 
incorporate the teachings of Mizokawa into modeling user's emotion and personality in 
a computer user interface of Breese et al. because that would have allowed users of 
Breese to use user evaluation, emotion recognition units to observe various pattern of 
behavior or modality, further allowing to determine pseudo-emotions, predetermined 
relationship between patterns of autonomous behavior, [see Mizokawa: col 2, 
line 1 5-34], thus improving evaluation of user's intellectual work or desire [col 1 , 
line 51-60]. 

8. As to Claim 2, Mizokawa teaches a system which including 'modifying at least 
one modifiable environmental unit to at least partially conform to the preferred 
modalities' [col 6, line 46-54, fig 4]. 

9. As to Claim 3, Breese teaches a system which including 'environment unit is 
modified in real-time' [col 1 7, line 35-43]. 

1 0. As to Claim 4, Mizokawa teaches a system which including 'storing the sensed 
psychomotor behavioral element in a user history' [col 7, line 32-40, line 51-55]. 
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11. As to Claim 5,13,1 5, Breese teaches algebraic transforms' [col 2, line 38-49], 
Breese specifically teaches Bayesian network model associate with various algebraic 
transforms. On the other hand, Mizokawa teaches a system which including 'sensed 
psychomotor behavioral element is stored' [col 7, line 53-59, col 8, line 14-29], 
psychomotor behavioral element corresponds to Mizokawa's user's emotions that 
including obedience, aggressiveness, curiosity, cheerful and like as detailed in col 8, 
line 14-29. 

12. As to Claim 6, the limitations of this claim have been noted in the rejection of 
claim 1 above. In addition, Mizokawa teaches determining preferred modalities includes 
determining a preferred combination of modalities and an ordering of modalities by 
preference thereby further defining a focus of the individual's attention' [col 7, line 11- 
17, line 21-24], Mizokawa specifically teaches sound/voice recognition, analyzing 
information on the user's gestures] 

1 3. As to claim 7, Mizokawa disclosed 'modifying the environmental unit to provide 
content in the environment in the preferred combination of modalities and the order of 
modalities by preference whereby the combination and the order are placed in at least 
one respective co-ordinate group of representational geometry to which attention of the 
individual is drawn, as indicated by the psychomotor behavioral element' [col 6, line 46- 
62]. 
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14. As to claim 8, Breese disclosed 'defining a psychodynamic and a cognitive 
behavioral model using preferred combination modalities and the order of modalities' 
[col 5, line 5-10, line 23-28]; 'modifying at least one environmental unit as a function of 
the psychodynamic behavioral model and the cognitive behavioral model' [col 8, line 35- 
43]. 

1 5. As to Claim 1 0, the limitations of this claim have been noted in the rejection of 
claim 1 above. And in addition, Breese teaches 'multi-dimensional and has a plurality of 
modifiable environmental units' [col 10, line 59-67]. 

1 6. As to claim 1 1 , Mizokawa disclosed 'preprogramming the device to monitor the 
individual for at least one specific types of psychomotor behavioral elements' [col 4, line 
25-46]; 'communicating an occurrence of the specific type of psychomotor behavioral 
element' [col 4, line 39-46]. 

17. As to claim 14, Mizokawa disclosed 'memory device to store sensed 
psychomotor behavioral activity of the individual' [col 7, line 60-64]. 

18. As to claim 16, Mizokawa disclosed 'modalities are calculated while sensing 
psychomotor behavioral activity and concurrently used for modifications to the 
environmental units' [col 10, line 15-24]. 
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1 9. As to claim 1 7, Mizokawa disclosed 'sensor includes at least one input device for 
a computer and the modifiable environmental unit includes at least one output device' 
[fig 1, col 4, line 51-59]. 

20. Claims 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Breese et al. [hereafter Breese], US Patent No. 5,987,415 filed on Ju 30,1998, 
published on Nov 16, 1999 in view of Mizokawa, US Patent No.6, 230,111 filed on 
Aug 6, 1998, published on May 8, 2001 

21. It is, noted both Breese, Mizokawa do not teach 'modalities calculation by an 
equation', although Breese teaches modeling a user's emotion and personality 
specifically using Bayesian network inference algorithm [see abstract, col 12, line 29-35], 
while Mizokawa teaches control system for controlling object using pseudo-emotions 
and personality specifically calculating using rules or functional equations for pseudo 
emotions levels and personalities [col 10, line 15-18]. Therefore, it would have been 
obvious to one of the ordinary skill in the art at the time of applicant's invention to not 
only sense emotions, but also establish relationship between emotions, environmental 
information to evaluate the behavioral output as suggested by Mizokawa 

[col 9, line 16-21]. 
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Response to Arguments 

22. Applicant's arguments in the appeal brief, filed on 9/1/2005, with respect to the 
rejection of claims 1-18 have been considered but are moot in view of the new 
ground(s) of rejection as stated above. 

Conclusion 
The prior art made of record 



a. 
b. 



US Patent No. 
US Patent No. 



5987415 
62301 1 1 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Srirama Channavajjala whose telephone number is 
571-272-4108. The examiner can normally be reached on Monday-Friday from 
8:00 AM to 5:30 PM Eastern Time. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Alam, Hosain, T, can be reached on (571) 272-3978. The fax phone 
numbers for the organization where the application or proceeding is assigned is 
571-273-8300 Information regarding the status of an application may be obtained 
from the Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. 
Should you have questions on access to the Private PAIR system, contact the 
Electronic Business Center (EBC) at 866-217-9197 (toll-free) 

sc jfay^ Alam, Hosain, T 

Patent Examiner. SPE.AU2166. 
October 14, 2005. 



